Video game controlled by player motion tracking

ABSTRACT

A rhythm-based video game (“game”) is disclosed. In the game, a player slashes blocks representing musical beats using a pair of energy blades resembling a lightsaber. A gaming console renders multiple digital objects, e.g., digital blocks, digital mines and digital obstacles, that are approaching a player in a virtual space. The gaming console also renders a digital representation of an instrument, e.g., a lightsaber (“digital saber”), using which the player slashes, cuts or otherwise interacts with the digital blocks to cause a digital collision between the digital saber and the digital blocks. The player can score by slashing the digital blocks, not hitting the digital mines and avoiding the digital obstacles. The game presents the player with a stream of approaching digital objects in synchronization with music, e.g., a song&#39;s beats, being played in the game. The pace at which the digital blocks approach the player increases with the beats.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.16/195,498, entitled “VIDEO GAME CONTROLLED BY PLAYER MOTION TRACKING,”and filed Nov. 19, 2018, which claims priority to U.S. ProvisionalApplication No. 62/755,468, entitled “VIDEO GAME INCLUDING BODY TRACKINGAND VIRTUAL OBJECTS, and filed Nov. 3, 2018, both of which areincorporated herein by reference for all purposes.

TECHNICAL FIELD

The disclosure is related to video games, and more specifically to videogames that track the movements of the player.

BACKGROUND

A player-tracking video game is one where a user's movements (e.g., oftheir head, body, arms, wrists, hands, or legs) provide meaningfulcontrols to the game. Player-tracking games take a variety of forms andoften include music video games.

A music video game is a video game where the gameplay is meaningfullyand often almost entirely oriented around the player's interactions witha musical score or individual songs. Music video games may take avariety of forms and are often grouped with puzzle games due to theircommon use of “rhythmically generated puzzles.” Music video games aredistinct from purely audio games in that they feature a visual feedbackto lead the player through the game's soundtrack, although eidetic musicgames can fall under both categories.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an environment in which a rhythm-basedvideo game can be implemented.

FIG. 2 is a block diagram of a three-dimensional (3D) virtual space inwhich the game is played, consistent with various embodiments.

FIG. 3 is a flow diagram of a process for presenting the game to aplayer, consistent with various embodiments.

FIG. 4 is a flow diagram of a process for presenting the game to aplayer, consistent with various embodiments.

FIG. 5 is a flow diagram of a process for presenting the game to aplayer, consistent with various embodiments.

FIG. 6 is a flow diagram of a process for presenting the game to aplayer, consistent with various embodiments.

FIG. 7 illustrates variable game actions based on angle and position ofincidence between a digital element and a digital object.

FIG. 8 is a screenshot of a player playing the game, consistent withvarious embodiments.

FIG. 9 is a screenshot of a player view of the 3D virtual spacegenerated by the gaming console, consistent with various embodiments.

FIG. 10 is a screenshot of a graphical user interface (GUI) withinstructions to the player, consistent with various embodiments.

FIG. 11 is a screenshot of a GUI with instructions to the player,consistent with various embodiments.

FIG. 12A is a screenshot of a GUI with instructions to the player,consistent with various embodiments.

FIG. 12B is a screenshot of a GUI with instructions to the player,consistent with various embodiments.

FIG. 13 is a screenshot of a GUI with instructions to the player,consistent with various embodiments.

FIG. 14 is a screenshot of a GUI in which the player can select variousoptions, consistent with various embodiments.

FIG. 15 is a screenshot of a GUI showing multiple digital objectsapproaching the player in the 3D virtual space, consistent with variousembodiments.

FIG. 16 is a screenshot of a GUI showing multiple digital objectsapproaching the player in the 3D virtual space, consistent with variousembodiments.

FIG. 17 is a diagram of a multi-lane game environment.

FIG. 18 is a screenshot of a GUI showing two lanes at different anglesabout the player, each lane including digital objects approaching theplayer in the 3D virtual space, consistent with various embodiments.

FIG. 19 is a screenshot of a GUI showing two parallel lanes of digitalobjects approaching the player in the 3D virtual space, consistent withvarious embodiments.

FIG. 20 is a screenshot of a GUI showing multiple lanes at differentangles of digital objects approaching the player in the 3D virtual spaceusing a static world center, consistent with various embodiments.

FIG. 21 is a block diagram of a computer system as may be used toimplement features of some embodiments of the disclosed technology.

DETAILED DESCRIPTION

The disclosure is related to a video game (“game”). In the game a playerinteracts with digital objects that approach the user in a 3D corridor.Interaction occurs via digital collision between a digital elementcontrolled by the player. Control of the digital element is based onbody tracking. In some embodiments, body tracking is performed via aworn or held peripheral that tracks its own movement relative some otherreference point. In some embodiments, a depth camera or room-mappingcameras (e.g., Playstation Camera, Microsoft Kinect, LeapMotion, orequivalents) captures video of the player and uses computer visiontechniques to identify body positions of the user. The game presents theplayer with a stream of approaching digital objects and the user causesthe digital element to collide with the digital objects based on use ofbody tracking. Digital collisions with different types of digitalobjects and from different angles cause a variety of game actions tooccur. In some embodiments, the game further tracks the motion of theuser's body and shifts a player location in game corresponding to bodymovements. Movement of the player location enables the player to avoiddigital obstacles.

An embodiment of the disclosure is related to a rhythm-based video game.In the rhythm-based video game a player slashes blocks representingmusical beats using one or a pair of energy blades resembling alightsaber (the digital element). A gaming console renders multipledigital objects, e.g., a digital representation of a block (“digitalblock” or “block”), that are moving in a specified direction, e.g., in adirection towards a user or player. The gaming console also renders adigital representation of an instrument, e.g., a digital representationof a sword resembling a lightsaber (“digital saber”), using which theplayer slashes, cuts or otherwise interacts with the digital blocks tocause a digital collision between the digital saber and the digitalblocks. The game presents the player with a stream of approachingdigital blocks in synchronization with beats of music, e.g., a song'sbeats and notes, being played in the game. As the beat picks up in themusic, the pace at which the digital blocks approach the player canincrease.

A game action occurs in response to the digital collision. The gameaction can be any of an increase or decrease in score of the player, anincrease or decrease in energy of the player, a gain or loss of life ofthe player in the game, an increase or decrease in a rate at which thescore changes, an increase or decrease in the pace at which the blocksmove towards the player, etc. The game can end based on multiplefactors, such as after a specified time period, when the player runs outof energy or lives, or when the player issues a command to stop thegame. The video game can be implemented as a two-dimensional (2D) videogame, a three-dimensional (3D) video game, a virtual reality (VR) game,or an augmented reality (AR) game. In some embodiments, the gamingconsole is configured to implement the video game as a VR game.

FIG. 1 is a block diagram of an environment 100 in which therhythm-based video game can be implemented. The environment 100 includesa gaming console 105 which executes a rhythm-based video game, such asthe one described above. In some embodiments, the gaming console 105 canbe a computing device having a processor and memory, and the processorexecutes instructions stored in the memory to present the game to aplayer 110 on a display device 115. The display device 115 supports 2Dand/or 3D rendering of the game. In some embodiments, the player 110 mayhave to wear 3D glasses (not illustrated) to experience the game in 3D.The gaming console 105 supports a VR implementation of the game. In theVR implementation, an apparatus such as a headset 120 may have to beused by the player 110 to experience the game in VR.

The headset 120 is head-mounted device, which is used to track theorientation or position of a body or head of the player 110. The headset120 has one or more display devices that presents the game in VR. Theheadset 120 can also have one or more sensors that are used to determineand transmit coordinates of the position of the player 110 to the gamingconsole 105. Examples of such sensors include gyroscopes,accelerometers, structured light systems, depth sensing cameras,magnetic position sensors, and eye tracking sensors. Sensors can belocated in one or more locations, e.g., integrated with the headset 120,be worn by the player 110 anywhere on the body, integrated with a motioncontroller 125, or part of other equipment worn by the player 110. Thegaming console 105 establishes the position of the player 110 in a 3Dvirtual space by translating the coordinates received from the headset120 to coordinates in the 3D virtual space. The coordinates receivedfrom the headset 120 can also help in determining different positions oractions of the player 110, e.g., whether the player 110 is sitting,standing, ducking, jumping, moving, etc. The headset 120 may include amicrophone to receive any audio input from the player 110 or thesurroundings of the player 110. The headset 120 may include one or morespeakers that outputs audio to the player 110, such as the song beingplayed in the game. The headset 120 can communicate with the gamingconsole 105 wirelessly or using wired means.

The environment 100 also includes a hand-held or hand-worn apparatussuch as a motion controller 125, which is used to track an orientation,position and movement of the hand of the player 110. The motioncontroller 125 includes one or more sensors, e.g., such as the onesmentioned above, that track the orientation, position and motion of thehand of the player 110 (examples include an Oculus Touch, a Samsung Gearcontroller, a PlayStation Move, or a Nintendo Switch controller). Insome embodiments, the motion controller includes a magnetic positionsensor that senses a position of the motion controller 125 in relationto a non-hand-held peripheral, e.g., headset 120. The motion controller125 transmits the coordinates of the hand and/or the movement of thehand to the gaming console 105, which then translates the movement tothe 3D virtual space. The motion controller 125 can also include ahaptic feedback mechanism that provides haptic feedback, e.g., when theplayer 110 slashes the digital block. In some embodiments, theenvironment 100 may include more than one motion controller 125, e.g., apair of motion controllers. The player 110 can hold one motioncontroller in one hand and the other in the other hand. In someembodiments, a single motion controller 125 is held in both hands. In atwo-player game one player can hold one motion controller and the otherplayer can hold the other motion controller. The motion controller 125can be of any shape, size or dimension that is suitable to be held inthe hand of a player. The motion controller 125 can communicate with thegaming console 105 wirelessly or using wired means. The motioncontroller 125 can also communicate with other devices, such as headset120, wirelessly or using wired means.

In the rhythm-based video game, the gaming console 105 establishes a 3Dvirtual space, such as the 3D virtual space 205 of FIG. 2. The 3Dvirtual space 205 includes a position of the player 110, e.g., which isdetermined based on the coordinates of the position of the player 110obtained from the headset 120. The 3D virtual space 205 includes aproximate end 210 that is proximate to the player 110 and a distal end215 opposite to the proximate end 210. A portion of the proximate end210 corresponds to the position of the player 110. In some embodiments,the player may calibrate the controller or the virtual space 205 tothemselves. Calibration may be performed by the hardware, or manually bythe player.

The gaming console 105 renders multiple digital objects, e.g., a digitalblock 220, that are approaching the player 110 from the distal end 220.In some embodiments, the digital block 220 may appear bigger in size asit approaches the proximate end 210. The gaming console 105 also rendersa digital representation of an instrument, e.g., a digital saber, usingwhich the player 110 can slash, cut or otherwise interact with thedigital block 220 to cause a game action to occur in the game. The gamepresents the player 110 with a stream of the digital blocks insynchronization with beats of music, e.g., a song's beats and notes,being played in the game. As the beat picks up in the music, the pace atwhich the digital blocks approach the player 110 can increase.

In the VR implementation, the motion controller 125 can be a VR basedmotion controller, which is represented as a digital saber in the 3Dvirtual space 205. The player 110 uses a pair of VR motion controllersto wield a pair of digital lightsabers, e.g., a first digital saber 230and a second digital saber 235, in the 3D virtual space 205 to slash thedigital blocks. The digital blocks can be of various types, e.g., afirst type and a second type, which the player 110 may interact withusing the two different digital sabers. A specific type of digitalblocks should be interacted with using a specified digital saber. Insome embodiments, the first type of digital blocks can be of a firstcolor and may have to be interacted with using a digital saber of thecorresponding color, and the second type of digital blocks can be of asecond color and may have to be interacted with using a digital saber ofthe corresponding color. For example, each digital block is colored redor blue to indicate whether the red or blue digital saber should be usedto slash it.

In some embodiments, each of the digital blocks is marked with adirection indicator 225, which indicates the direction to slash throughthe digital block. For example, a directional indicator 225 such as anarrow can indicate one of eight possible directions to slash through thedigital block 220. In another example, a directional indicator 225 suchas a dot can indicate that the player 110 may slash through the digitalblock 220 in any direction. When a digital block is slashed by a digitalsaber, the digital block is destroyed, and a score is awarded based onone or more factors, e.g., timing accuracy and physical positioning ofthe cut.

The game can also present digital objects other than digital blocks,which the player 110 should not hit. For example, the game can present adigital object such as a “digital mine” 240 that the player 110 shouldnot hit. In another example, the game can present obstacles such as adigital representation of an oncoming wall (“digital obstacle”) 245 thatthe player 110 or the head of the player 110 should avoid. The player110 can avoid the digital obstacle 245 approaching the proximate end 210by moving out of the path of digital obstacle 245, which can be done bystepping to the right or left of the digital obstacle 245 or by duckingbelow the digital obstacle 245. For example, if the player 110 steps tothe right (in the real world) of the oncoming digital obstacle 245, thegaming console 105 shifts at least a portion of the 3D virtual space 205to the right of the player 110 so that the digital obstacle ends uppassing through the left of the player 110 at the proximate end 210.

The sensors worn by the player 110, e.g., in the headset 120, motioncontroller 125 or elsewhere, can transmit the coordinates of the player110, portion of the body of the player 110, such as a head of the player110, movements of the player 110, or movements of a portion of the bodyof the player 110 to the gaming console 105. The gaming console 105translates the received coordinates to the coordinates in the 3D virtualspace 205 and determines the action of the player 110, e.g., whetherthere was a digital collision between a digital block and a digitalsaber, whether there was a digital collision between a digital obstacleand a digital saber, whether there was a digital collision between adigital wall and any portion of the body of the player 110 (whichcorresponds to a portion of the 3D virtual space in the proximate end210), whether the player 110 moved out of the path of the approachingdigital obstacles, etc., which result in a specified game action.

A game action can occur in response to a digital collision between adigital saber and a digital object. The game action can be any of anincrease or decrease in score of the player, an increase or decrease inenergy of the player 110, a gain or loss of life of the player 110 inthe game, an increase or decrease in a rate at which the score changes,an increase or decrease in the pace at which the digital objects arecreated or move towards the player 110, etc. Different types of gameactions can occur in response to different events. For example, a scoreof the player 110 can increase in response to a digital collisionbetween a digital block of a specified color and the digital saber ofthe specified color. In another example, a score of the player 110 candecrease in response to a digital collision between a digital block of aspecified color and a digital saber of a color other than the specifiedcolor. In another example, a score of the player 110 may increase, orincrease by an amount above a threshold, in response to a digitalcollision between a digital block and a digital saber in which a contactangle of the digital saber with the digital block is consistent with thedirection indicated on the digital block.

In another example, a score of the player 110 may not increase, orincrease by an amount below a threshold, in response to a digitalcollision between a digital block and a digital saber in which a contactangle of the digital saber with the digital block is not consistent withthe direction indicated on the digital block. In another example, ascore of the player 110 may not increase, or increase by an amount belowa threshold, in response to a digital collision between a digital blockand a digital saber whose collision impact is below a specifiedthreshold.

In some embodiments, the collision impact can be measured as a functionof how hard, fast or strong the player 110 swings the motion controller125 to slash the digital block 220. In another example, a score, energyor life of the player 110 can decrease in response to a digitalcollision between a digital mine and a digital saber. In anotherexample, a score, energy or life of the player 110 can decrease inresponse to a digital collision between a digital obstacle and a digitalsaber or the player 110. In some embodiments, a game action can alsooccur if there is no digital collision for a specified duration. Forexample, if the player 110 does not slash through any of the digitalblocks for a duration exceeding a specified threshold, a score, energyor life of the player 110 can decrease, or the rate at which the score,energy or life increases can be decreased. The game can be configured tocalculate the score, energy, or a life of the player 110 using variousfactors, including the above specified factors.

The game can end based on one or more factors, such as after a specifiedtime period, when the player runs out of energy or lives, when theplayer 110 has completed one or more levels of the game, when the musicstops playing, or when the player issues a command to stop the game.

As mentioned above, the game can be implemented as a 2D game, 3D game, aVR game, or an AR game. The entities of the environment 100 can beadjusted, adapted or configured for a specific implementation. Forexample, while the environment 100 described providing the VR gamingexperience through the headset 120, in some embodiments, the VR gamingexperience can also be created through specially designed rooms withmultiple large screens.

In some embodiments, some of the entities of the environment 100 mayhave to be calibrated before being able to play the game to obtain thecoordinates of the position of the player 110. For example, a setupprocess in the game may ask the player 110 to turn the headset 120 andperform some gestures, such as moving from left to right, right to left,ducking, jumping, or talking. In another example, the setup process inthe game may ask the player 110 to move the motion controller in eachhand to perform a gesture, such as raising the motion controller,swinging the hand with the motion controller to hit a digital objectdisplayed on the display device 115 or the headset 120. Once thecalibration is complete, the gaming console 105 establishes the 3Dvirtual space 205 after which the player 110 can proceed to play thegame.

In some embodiments, the user is able to customize their gameexperience. Examples include changing the graphical representation onthe digital element (the digital sabers) 230/235. The digital sabers230/235 may change color, change in graphical design and through use ofvarious “skins.” The sabers 230/235 may also change in shape orcharacter causing the manner in which the user causes digital collisionsto shift. In some embodiments, a player avatar is displayed to the user.The player avatar is customizable using skins and different digitalmodels. In some embodiments, the user is able to generate gameplay viaattaching a “beat map” to an audio file. A beat map includes datadescribing each digital object 220,240,245 that is generated in the 3Dvirtual space 205, at what point in the audio file the objects220,240,245 are generated, the speed of the objects 220,240,245, thetype/color of each object 220,240,245, the directionality 225 of eachobject 220,240,245, and a position and vector in the 3D virtual space205 of each object 220,240,245. Given a beat map and a correspondingaudio file, any song can be played in the game. A digital distributionsystem may also provide packs or groups of beat maps and audio files toplay with the game.

FIG. 3 is a flow diagram of a process 300 for presenting the game to aplayer, consistent with various embodiments. In some embodiments, theprocess 300 can be implemented in the environment 100 of FIG. 1. Theplayer 110 can complete the setup process to calibrate the motioncontroller and the headset 120. In some embodiments, the player 110 mayneed to complete the calibration only once per session, e.g., when thegaming console 105 is powered on and prior to playing a first game andneed not calibrate the devices again unless the gaming console 105 ispowered off and powered on again. In some embodiments, the devices mayhave to be calibrated when the player changes. In some embodiments, thedevices may have to be calibrated when there are environmental changesaround the player 110, e.g., a change in intensity of light in the roomwhere the player 110 plays the game, a change in distance between theplayer 110 and the gaming console 105, a change in the position of thefurniture in the room where the game is played. After the devices arecalibrated, the gaming console 105 will have the necessary coordinatesto establish the 3D virtual space in which the game is played.

At block 305, the gaming console 105 renders the 3D virtual space 305 inwhich the game is to be played. The 3D virtual space 305 includes adigital position of the player 110.

At block 310, the gaming console 105 renders a digital element, e.g., adigital saber, in the 3D virtual space 305 relative to the digitalposition of the player 110. The coordinates and the orientation of thedigital saber relative to the digital position correspond to anorientation and the coordinates of the physical hand of the player 110relative to the physical body of the player 110. The gaming console 105obtains the orientation and the coordinates of the physical hand of theplayer 110 relative to the physical body of the player 110 using themotion controller 125 held in the hand of the player 110 and the headset120 worn by the player 110.

At block 315, the gaming console 105 renders multiple digital objectsthat approach the digital position of the player 110 from a distance inthe 3D virtual space. The digital objects can include digital blocksthat the player 110 should slash using the digital element. In someembodiments, the digital objects can include digital mines that theplayer 110 should not hit, and digital obstacles that the player 110should avoid.

At block 320, the gaming console 105 causes a game action to occur inresponse to a digital collision between the digital element and one ormore of the digital objects. The different type of game actions that canoccur are described at least with reference to FIG. 1.

FIG. 4 is a flow diagram of a process 400 for presenting the game to aplayer, consistent with various embodiments. In some embodiments, theprocess 400 can be implemented in the environment 100 of FIG. 1. Theprocess 400 assumes that the headset 120 and the motion controllers arecalibrated. At block 405, the gaming console 105 renders a 4D virtualspace in which the game is to be played. The 4D virtual space 205includes a proximate end 210 that is proximate to the player 110 and adistal end 215 opposite to the proximate end 210. In some embodiments, aportion of the proximate end 210 corresponds to the digital position ofthe player 110.

At block 410, the gaming console 105 renders a digital representation ofan instrument to be used by the player 110, e.g., the first digitalsaber 230, to play the game. The coordinates and the orientation of thefirst digital saber 230 relative to the digital position of the player110 correspond to an orientation and the coordinates of the physicalhand of the player 110 relative to the physical body of the player 110.

At block 415, the gaming console 105 instantiates multiple digitalblocks in the 4D virtual space 205.

At block 420, the gaming console 105 associates each of the digitalblocks with a direction indicator. In some embodiments, the directionindicator indicates the direction in which the digital block is to beslashed by the player 110.

At block 425, the gaming console 105 plays an audio file having amusical beat, e.g., a song.

At block 430, the gaming console 105 causes the multiple digital objectsto travel from the distal end 215 to the proximate end 210 in the 4Dvirtual space 205 in synchronization with the musical beats. Forexample, the rate at which the digital blocks are created or the pace atwhich the digital blocks approach the proximate end 210 depend on themusical beats. As the beat picks up in the song, the pace at which thedigital blocks approach the player can increase.

At block 435, the gaming console 105 causes a game action to occur basedon an interaction between the first digital saber 230 and one or more ofthe digital blocks. The different type of game actions that can occurare described at least with reference to FIG. 1.

FIG. 5 is a flow diagram of a process 500 for presenting the game to aplayer, consistent with various embodiments. In some embodiments, theprocess 500 can be implemented in the environment 100 of FIG. 1. Theprocess 500 assumes that the headset 120 and the motion controllers arecalibrated. At block 505, the gaming console 105 renders a 3D virtualspace in which the game is to be played. The 3D virtual space 205includes a proximate end 210 that is proximate to the player 110 and adistal end 215 opposite to the proximate end 210. In some embodiments, aportion of the proximate end 210 corresponds to the digital position ofthe player 110.

At block 510, the gaming console 105 renders a digital representation ofinstruments to be used by the player 110, e.g., the first digital saber230 and the second digital saber 235, to play the game. For example, thefirst digital saber 230 can correspond to the motion controller held bythe player 110 in the left hand and the second digital saber 235 cancorrespond to the motion controller held by the player 110 in the righthand. The coordinates and the orientation of the digital sabers relativeto the digital position of the player 110 correspond to an orientationand the coordinates of the physical hands of the player 110 relative tothe physical body of the player 110. The digital sabers can havedifferent characteristics. For example, the first digital saber 230 canbe a red colored saber and the second digital saber 235 can be a bluecolored saber.

At block 515, the gaming console 105 renders multiple digital objectstraveling in the 3D virtual space 205 from the distal end 215 to theproximate end 210. The digital blocks can include two different sets ofblocks. In some embodiments, the first set of digital blocks can be of afirst color and the second set of digital blocks can be of a secondcolor. For example, each digital block is colored red or blue toindicate whether the red or blue digital saber should be used to slashit.

At determination block 520, the gaming console determines whether therewas an interaction, e.g., digital collision, between the red digitalblocks and the red saber 230 or between the blue digital blocks and theblue saber 235.

If the gaming console 105 determines that at least one of the conditionsin block 520 is true, at block 525, the gaming console 105 causes afirst type of game action, and the control is transferred to block 515.For example, the first type of game action can be to increase a score ofthe player 110 in response to a digital collision between a digitalblock and the digital saber of the same color.

At determination block 530, the gaming console determines whether therewas an interaction, e.g., digital collision, between the red digitalblocks and the blue saber 235 or between the blue digital blocks and thered saber 230.

If the gaming console 105 determines that at least one of the conditionsin block 530 is true, at block 535, the gaming console 105 causes asecond type of game action, and the control is transferred to block 515.For example, the second type of game action can be not to increase thescore, or decrease the score of the player 110 in response to a digitalcollision between a digital block and the digital saber of differentcolors.

FIG. 6 is a flow diagram of a process 600 for presenting the game to aplayer, consistent with various embodiments. In some embodiments, theprocess 600 can be implemented in the environment 100 of FIG. 1. Atblock 605, the gaming console 105 obtains an orientation and coordinatesof the player 110, a hand and head of the player 110 at least from theheadset 120 and the motion controller 125.

At block 610, the gaming console 105 renders a 3D virtual space in whichthe game is to be played. The 3D virtual space 205 is generated based onthe coordinates received from the headset 120 and the motion controller125 associated with the player 110. The 3D virtual space 205 includes aproximate end 210 that is proximate to the player 110 and a distal end215 opposite to the proximate end 210. In some embodiments, a portion ofthe proximate end 210 corresponds to the digital position of the player110.

At block 615, the gaming console 105 renders a digital representation ofinstruments to be used by the player 110, e.g., the first digital saber230 and the second digital saber 235, at the proximate end 210. Thedigital saber is responsive to the movements of the hand of the player110 holding the motion controller 125. The coordinates and theorientation of the digital sabers relative to the digital position ofthe player 110 correspond to an orientation and the coordinates of thephysical hands of the player 110 relative to the physical body of theplayer 110.

At block 620, the gaming console 105 renders multiple digital objectstraveling in the 3D virtual space 205 from the distal end 215 to theproximate end 210. The digital objects can be of different types. Forexample, a first type can include digital blocks that the player 110should slash using the digital saber, a second type can include digitalmines that the player 110 should not hit, and a third type can includedigital obstacles that the player 110 should avoid.

At block 625, the gaming console 105 causes interaction with the digitalobjects based on the movement of the hand, head and/or body of theplayer 110.

At determination block 630, the gaming console 105 determines whetherthere was an interaction, e.g., digital collision, between a digitalblock and the digital saber. If yes, at block 635, the gaming console105 causes a first type of game action. For example, the first type ofgame action can be to increase a score of the player 110 in response toa digital collision between a digital block and the digital saber. Thecontrol is transferred to block 625.

At determination block 640, the gaming console 105 determines whetherthere was an interaction, e.g., digital collision, between the digitalsaber and a digital mine. If yes, at block 645, the gaming console 105causes a second type of game action. For example, the second type ofgame action can be not to increase the score or decrease thescore/energy/life of the player 110 in response to the digital collisionbetween a digital mine and the digital saber. The control is transferredto block 625.

At block 650, the gaming console 105 causes the digital obstacle tochange the direction of travel based on actual movements of the player110 or movements of the head of the player 110.

At determination block 655, the gaming console 105 determines whetherthe digital obstacle passes through the digital position of the player110 at the proximate end 210. If yes, at block 660, the gaming console105 causes a third type of game action. For example, the third type ofgame action can be to decrease the score/energy/life of the player 110.The control is transferred to block 625.

FIG. 7 illustrates variable game actions based on angle and position ofincidence between a digital element 730 and a digital object 720. Thepictured digital object 720 includes a direction indicator 725. In theillustrated example, the digital element 730 is elongated and extendsfrom a control point at the user's hand (as a sword would).

The direction indicator 275 indicates the game's ideal collisiondirection between the digital element 730 and the digital object 720.Causing a collision using a swing direction 735 that corresponds to thedirection indicator 725 results in a positive game action (e.g.,rewarding of points), whereas causing a collision at a differentdirection causes a different game action (e.g., issuing a fault, endinga combo streak, rewarding fewer points than the positive game action,subtracting points).

In some embodiments, angle incidence and/or digital element rotation 740may affect the type of game action occurring from a collision. Incidenceangles 745, 750 and 755 illustrate regions an incoming digital element730 might collide with the digital object 720. In some embodiments, thegame action occurring from a collision is more positive for the userwhen the collision has an incidence angle 745, 750, 755 closest toon-center (e.g., straight through center “mass” of the digital object720). For example, the collision may be worth more points depending onwhere a user strikes the digital object 720 with the digital element730.

The incidence angles 745, 750,755 may be used as a region (e.g., plus orminus 0-30 degrees from center) or as an absolute measurement (e.g.,exactly 37 degrees right or left of center) where a collision at 0degrees from center is worth the most points. In some embodiments,instead of, or in addition to incidence angles, entry and exit surfacesare used. Where a collision begins and ends on opposite sides of adigital object 720, the digital element 730 took a relatively straightpath through the digital object 720. Taking a straight path through thedigital object 720 may provide users with a more positive game action(e.g., more points) than a collision that enters and exits throughadjacent sides of the digital object 720. Illustrative of the relativelystraight path described above is a collision “cut” that begins onsurface 760 and exits through surface 765. A collision that does not usea straight path begins at surface 760 and exits through surface 770; oralternatively, a collision that begins at surface 770 and exits throughsurface 765. The non-straight path collisions may render less positivegame actions than straight path collisions.

In some embodiments, the digital element rotation 740 further influencesgame actions. The digital element rotation 740 is controlled by theuser's motion controller or hand gestures. Where the user rotates withinthe applicable control scheme, the digital element 730 will rotate ingame, in a corresponding fashion. Some embodiments of digital elements730 have varied surfaces. Examples of varied surfaces include a bladededge of a sword and the flat of a sword. In this example, where a userstrikes the digital object 720 with a bladed edge (e.g., a cut), adifferent game action results than if the user strikes the digitalobject 720 with the flat side (e.g., a slap). In some embodiments, a cutrenders a more positive game action than a slap.

FIGS. 8-16 illustrate screenshots of various graphical user interfaces(GUIs) of the game, which are generated by the gaming console 105 ofFIG. 1.

FIG. 8 is a screenshot of a player playing the game, consistent withvarious embodiments. In FIG. 8, the player is holding a pair of motioncontrollers in the hands which are depicted as digital sabers in the 3Dvirtual space of the game. FIG. 8 also illustrates the player slashing apair of digital blocks.

FIG. 9 is a screenshot of a player view of the 3D virtual spacegenerated by the gaming console, consistent with various embodiments.

FIG. 10 is a screenshot of a graphical user interface (GUI) withinstructions to the player, consistent with various embodiments. In FIG.10, the GUI instructs the player to move their hands so that the digitalsabers cut the digital blocks in the indicated direction.

FIG. 11 is a screenshot of a GUI with instructions to the player,consistent with various embodiments. In FIG. 11, the GUI indicates tothe player that some of the digital objects presented in the 3D virtualspace are meant to be avoided, e.g., by not cutting or slashing theobjects.

FIG. 12A is a screenshot of a GUI with instructions to the player,consistent with various embodiments. In FIG. 12A, the GUI indicates tothe player that some of the digital objects presented in the 3D virtualspace are to be avoided by player's location as opposed to avoidingusing the digital sabers. For example, the player can duck, crouch ormove to avoid to some of the digital objects.

FIG. 12B is a screenshot of a GUI with instructions to the player,consistent with various embodiments. In FIG. 12B, the GUI indicates tothe player that some of the digital objects presented in the 3D virtualspace are to be avoided by ducking or crouching.

FIG. 13 is a screenshot of a GUI with instructions to the player,consistent with various embodiments. In FIG. 13, the GUI indicates tothe player that the player can gain points by cutting the correctdigital objects and in the correct manner.

FIG. 14 is a screenshot of a GUI in which the player can select variousoptions, consistent with various embodiments. In FIG. 14, the GUIpresents the player with various songs to select from and a difficultylevel of the game. Note that the digital objects are presented in the 3Dvirtual space based on the beats in the music.

FIG. 15 is a screenshot of a GUI showing multiple digital objectsapproaching the player in the 3D virtual space, consistent with variousembodiments.

FIG. 16 is a screenshot of a GUI showing multiple digital objectsapproaching the player in the 3D virtual space, consistent with variousembodiments. In FIG. 16, the game presents digital blocks and digitalobstacles simultaneously. The player has to slash the digital blocks butavoid the digital obstacles.

FIG. 17 is a diagram of a multi-lane game environment 1700. In someembodiments, multiple corridors or lanes 1702 exist around the playerexisting at angular positions around a circle or sphere. The multi-laneenvironment 1700 may be implemented in virtual or augmented reality andmake use of an apparatus that tracks angular motion of the player's head(e.g., roll, pitch, and yaw), such as a head mounted display or externalmonitoring system. Each lane 1702 includes multiple vectors or traversalpaths 1704 that digital objects are spawned and programed to progressalong toward a player space 1706. The game enables the player 1708 totranslate position (e.g., motion in the X, Y, or Z axes) within theplayer space 1706 (e.g., so as to dodge incoming obstacles). In someembodiments, the player space 1706 is centered around a static anchorpoint 1710 from which each lane 1702 radiates out from.

While the player 1708 may translate through the player space 1706,digital objects continue to approach, or traverse the player space 1706relative to, the static anchor point 1710. A positive game action occurswhen the player 1708 is able to cause a collision at a particularcollision distance 1712 between certain digital objects and theirrespective, player-controlled digital element. The positioning of thecollision distance 1712 where the player initiates collisions with thedigital objects is measured as a distance from the player's position1708. In some embodiments, the collision distance 1712 from the playeris a function of the digital elements wielded by the player. Where thedigital elements are swords, the length of the collision distance 1712may be a function of the length of the sword.

As the player 1708 turns their head (e.g., through roll, pitch, and/oryaw movement) a current or active lane 1702A of objects is engaged. Insome embodiments, the digital elements collide only with digital objectsgenerated in the active lane 1702A. In other embodiments, collisions mayoccur across any of the lanes. Different game actions may result (e.g.,differing quantities of points awarded) for collisions occurring innon-active lanes as opposed to the active lane 1702A.

In some embodiments, the lanes 1702 exist at all times within the gameenvironment whether or not the lanes 1702 are presently being displayed.For example, the virtual environment may include the lanes 1702 as dataconstructs while they remain dormant (e.g., do not have active digitalobjects) and activate display of the lanes 1702 when a digital object isgenerated to progress down a traversal path 1704 of the lane 1702.Alternatively, the lanes 1702 may be generated in response to a call togenerate a digital object that would necessarily require a traversalpath 1704 of a lane 1702 in order to progress toward the relevant endpoint.

In some embodiments, digital objects approach, or traverse relative to,the position of the player 1708. Because the player is enabled totranslate through the player space 1706, traversal of the digitalobjects may appear to take non-linear or arcing paths that “fall into”the player's position 1708 as if trapped by a virtual gravity well ormagnetic field.

Lanes 1702 for digital objects may exist 360 degrees about the anchorpoint 1710, or player position 1708, and 360 degrees from a virtualhorizon. In some embodiments, lanes 1702 intersect the player space 1706(e.g., do not pass directly through the player position or the anchorpoint).

FIG. 18 is a screenshot of a GUI showing two lanes 1802 at differentangles about the player, each lane including digital objects 1804approaching the player in the 3D virtual space, consistent with variousembodiments. The active lane 1802A is pictured on the left and isindicated by a lane guide 1806. In the screenshot, the player ispreparing to switch active lanes by turning their head (yaw) to thenon-active lane 1802B to the right. In some embodiments, the non-activelane(s) 1802B is indicated to the player through their peripheral visionin the display of the virtual player space (e.g., through directdisplay). Optionally, additional visual guides displayed at the edges ofthe player's vision indicating that the player should either turn theirhead or prepare to turn their head.

As the player's head turns and centers vision on a lane, the active lanechanges (and the lane guide 1806 shifts to the new active lane). In someembodiments, the distance the player must turn their head in order totrigger a change in the active lane 1802A is based on the degrees fromthe active lane. For example, if the next lane from the active lane isforty-five degrees to the right, the amount the player must turn theirhead is a function of forty-five degrees (e.g., various embodimentsrequire a 20%, 50%, 51%, 60%, 75%, 90%, 100%, etc. degrees of turning ofthe total to trigger the shift in lane).

FIG. 19 is a screenshot of a GUI showing two parallel lanes 1902A, B ofdigital objects 1904 approaching the player in the 3D virtual space,consistent with various embodiments. Lanes 1902 may intersect the playerspace without intersecting with either the player's position or ananchor point. For example, two parallel lanes 1902A, B may straddle theanchor point (not pictured).

In some embodiments, where there are two parallel lanes 1902A, B, therestill exists an active lane 1902A and a non-active lane 1902B. Theactive lane 1902A is denoted by the lane guide 1906. The player position(not shown) may be implied from the screenshot by the positioning of thedigital element 1908.

FIG. 20 is a screenshot of a GUI showing multiple lanes 2002 atdifferent angles of digital objects 2004A, B approaching the player inthe 3D virtual space using a static world center (not pictured),consistent with various embodiments. Pictured here, a large number oflanes 2002 are present in close proximity requiring the player to actquickly in each lane 2002 with both translational movement to the left,and element-object collisions (occurring in opposite directions in eachlane). The player attempts to cause collisions with the digital cubes2004A, while physically dodging the digital obstructions 2004B (byremaining on the left side of each lane/radius to the static anchorpoint/world center). The active lane 2002A is pictured to the right sideand indicated by the lane guide 2006. The non-active lanes 2002B are tothe left of the active lane 2002A.

Because the lanes 2002 are in close proximity, the range of headrotation required to trigger a shift in the active lane 2002A is lowerthan was pictured in FIG. 18.

FIG. 21 is a block diagram of a computer system as may be used toimplement features of some embodiments of the disclosed technology. Thecomputing system 2100 may be used to implement any of the entities,components, modules, interfaces, or services depicted in the foregoingfigures (and in this specification). The computing system 2100 mayinclude one or more central processing units (“processors”) 2105, memory2110, input/output devices 2125 (e.g., keyboard and pointing devices,display devices), storage devices 2120 (e.g., disk drives), and networkadapters 2130 (e.g., network interfaces) that are connected to aninterconnect 2115. The interconnect 2115 is illustrated as anabstraction that represents any one or more separate physical buses,point to point connections, or both connected by appropriate bridges,adapters, or controllers. The interconnect 2115, therefore, may include,for example, a system bus, a Peripheral Component Interconnect (PCI) busor PCI-Express bus, a HyperTransport or industry standard architecture(ISA) bus, a small computer system interface (SCSI) bus, a universalserial bus (USB), IIC (I2C) bus, or an Institute of Electrical andElectronics Engineers (IEEE) standard 1394 bus, also called “Firewire”.

The memory 2110 and storage devices 2120 are computer-readable storagemedia that may store instructions that implement at least portions ofthe described technology. In addition, the data structures and messagestructures may be stored or transmitted via a data transmission medium,such as a signal on a communications link. Various communications linksmay be used, such as the Internet, a local area network, a wide areanetwork, or a point-to-point dial-up connection. Thus, computer-readablemedia can include computer-readable storage media (e.g.,“non-transitory” media) and computer-readable transmission media.

The instructions stored in memory 2110 can be implemented as softwareand/or firmware to program the processor(s) 2105 to carry out actionsdescribed above. In some embodiments, such software or firmware may beinitially provided to the processing system 2100 by downloading it froma remote system through the computing system 2100 (e.g., via networkadapter 2130).

The technology introduced herein can be implemented by, for example,programmable circuitry (e.g., one or more microprocessors) programmedwith software and/or firmware, or entirely in special-purpose hardwired(non-programmable) circuitry, or in a combination of such forms.Special-purpose hardwired circuitry may be in the form of, for example,one or more ASICs, PLDs, FPGAs, etc.

Although the invention is described herein with reference to thepreferred embodiment, one skilled in the art will readily appreciatethat other applications may be substituted for those set forth hereinwithout departing from the spirit and scope of the present invention.Accordingly, the invention should only be limited by the Claims includedbelow.

1. A method comprising: rendering a 3D virtual space that includes adigital position of a user, a world center, and a plurality of lanes,the plurality of lanes radiate from or traverse the 3D virtual spacerelative to the world center and each include one or more traversalpaths, and the digital position of the user is tracked in 6 degrees offreedom relative to the world center; instantiating in the 3D virtualspace a first digital element, wherein a set of coordinates andorientation of the first digital element relative to the digitalposition of the user corresponds to an orientation of a first real handof the user to a real body of the user; graphically representing aplurality of digital objects in the 3D virtual space that approach theworld center from a distance along one or more of the traversal paths ofthe plurality of lanes; and causing a game action to occur in responseto a digital collision in the 3D virtual space between a first digitalobject of the plurality of digital objects and the first digitalelement.
 2. The method of claim 1, wherein a pace of the approach of thedigital objects to the digital position of the user corresponds withmusical notes.
 3. The method of claim 1, further comprising: causing asecond game action to occur in response to a second digital object ofthe plurality of digital objects passing by a collision region withoutdigitally colliding with the first digital element, wherein thecollision region is positioned as a function of the digital position ofthe user.
 4. The method of claim 1, wherein the first digital objectincludes a directional guide component indicating a direction of userinteraction with the first digital object.
 5. The method of claim 1,wherein the game action is any of: increasing a game score; decreasingthe game score; effecting a pace of the approach of the digital objects;effecting a creation of the digital objects; or effecting a rate ofchange in the game score.
 6. The method of claim 1, further comprising:engaging an active lane of the plurality of lanes based on a position ofa user's head, wherein the first digital element is enabled to onlycollide with digital objects present on the active lane.
 7. The methodof claim 1, wherein each of the plurality of lanes includes a pluralityof traversal paths, wherein each digital object is associated with oneor more of the plurality of traversal paths, and a first traversal paththat the first digital object is associated with is dispositive of amanner in which the first digital object progresses toward the worldcenter.
 8. The method of claim 1, further comprising: instantiating inthe 3D virtual space a second digital element, wherein a second set ofcoordinates and orientation of the second digital element relative tothe digital position of the user corresponds to an orientation of asecond real hand of the user to the real body of the user; causing anadditional game action to occur in response to a digital collision inthe 3D virtual space between a second digital object of the plurality ofdigital objects and the second digital element.
 9. The method of claim8, wherein a first subset of the plurality of digital objects correspondto the first digital element and a second subset of the plurality ofdigital objects correspond to the second digital element.
 10. The methodof claim 1, further comprising: displaying to the user a graphic guideindicating that digital objects are progressing toward the world centeron a first lane of the plurality of lanes that is outside a user'scurrent field of view.
 11. A system comprising: a hand trackingapparatus configured to track a position and orientation of at least afirst real hand of a user; a processor configured to direct rendering a3D virtual space that includes a digital position of the user, a worldcenter, and a plurality of lanes, the plurality of lanes radiate from ortraverse the 3D virtual space relative to the world center and eachinclude one or more traversal paths, and the digital position of theuser is tracked in 6 degrees of freedom relative to the world center,wherein the processor is further configured to instantiate a firstdigital element in the 3D virtual space that positioned and orientedrelative to the digital position corresponding to the position andorientation of the first real hand of a user, wherein the processor isfurther configured to cause a display to graphically represent aplurality of digital objects in the 3D virtual space that approach theworld center from a distance along one or more of the traversal paths ofthe plurality of lanes and cause a game action to occur in response to adigital collision in the 3D virtual space between a first digital objectof the plurality of digital objects and the first digital element. 12.The system of claim 11, wherein the hand tracking apparatus is any of:hand-held peripherals that wirelessly transmit the position andorientation to a communication apparatus communicatively coupled withthe processor; hand-held peripherals including magnetic position sensorsthat sense a position of the hand-held peripherals in relation to anon-hand-held peripheral; or a depth sensing camera positioned toward atleast the first hand of the user.
 13. The system of claim 11, whereinthe 3D virtual space is any of: an immersive virtual reality environmentwherein the 3D space surrounds the user; a digital space overlaid on areal space via augmented reality wherein the 3D space surrounds theuser; or a digital space wherein a user perspective is positionedexternal to a fourth wall.
 14. The system of claim 11, furthercomprising: a body or head tracking apparatus configured to track bodyor head position and orientation, and wherein a detected change in bodyor head position and orientation cause the processor to effect acorresponding change in a position and orientation of the digitalposition of the user in the 3D virtual space.
 15. The system of claim14, wherein the processor is further configured to engage an active laneof the plurality of lanes based on a position of a user's head asdetected by the head tracking apparatus, wherein the first digitalelement is enabled to only collide with digital objects present on theactive lane.
 16. A method comprising: instantiating a 3D game space thatincludes a player location, and a plurality of 3D lanes, each 3D laneincludes one or more traversal paths that digital objects progress alongfrom a location at the edges of the 3D game space toward a target in theinterior of the game space; tracking the player location based on areal-world position of a user's head in six degrees of freedom; trackinga player hand location based on a real-world position of a user's handin six degrees of freedom, wherein a digital element corresponds to theplayer hand location; engaging an active lane of the plurality of 3Dlanes based on a field of view of the player location, wherein thedigital element interacts with digital objects associated with theactive lane; detecting a collision between a first digital objectassociated with the active lane and the digital element controlled bythe user, wherein the collision causes a game action
 17. The method ofclaim 16, wherein the target in the interior of the game space is anyof: A static 3D game space center; The player position; or A positiondetermined relative to either the static 3D game space center or theplayer position.
 18. The method of claim 16, further comprising: playinga musical score including musical notes, wherein a pace of movement ofthe plurality of digital objects through the 3D lane is based on themusical notes.
 19. The method of claim 16, further comprising:displaying only a subset of the plurality of 3D lanes, wherein thesubset includes those 3D lanes that include an active digital object.20. The method of claim 16, wherein each of the plurality of 3D lanesincludes a plurality of traversal paths, wherein each digital object isassociated with one or more of the plurality of traversal paths, and afirst traversal path that the first digital object is associated with isdispositive of a manner in which the first digital object progressestoward the world center.
 21. The method of claim 16, wherein at least afirst subset of the plurality of digital objects include a directionalguide component indicating a direction of user interaction with thefirst digital object, and wherein detected collision between the firstsubset of the plurality of digital objects and the digital element thatoccur consistently with the directional guide component effect a firstgame action and detected collisions between the first subset of theplurality of digital objects and the digital sword elements that occurinconsistently with the directional component effect a second gameaction.